Nearly all software installed on a computer system is updated at some point in time. Unfortunately, as those skilled in the art will appreciate, updating any software introduces the possibility of regression, i.e., the introduction of flaws and/or errors, generally referred to as “bugs”, that were not previously evident or present in a prior version.
While there are numerous automated testing systems to determine whether there is regression in an updated version of a typical software application, verifying whether there is regression in an updated version of a graphics rendering engine or graphics interface driver is a very difficult task.
A graphics rendering engine is a key component of most operating system environments. The graphics rendering engine provides a generic drawing interface to software applications, i.e., a generic drawing interface that is independent of the particular graphics interface installed on the host computer. The graphics rendering engine takes the generic drawing instructions it receives, converts them to the specific interface of the graphics interface, or its graphics interface driver installed on the host computer system, and transfers them to the graphics interface driver. FIG. 1 is a pictorial diagram illustrating the flow of graphical information from an application to a display device, as found in the prior art. In particular, an application 102 generates generic drawing instructions and data 104 directed at the generic interface provided by the graphics rendering engine 106. The graphics rendering engine 106 translates the generic instructions 104 to device-specific instructions 108 suitable for the graphics display subsystem 110, comprising a graphics interface driver 112 (software interface to the graphics interface hardware), the graphics interface 114, and the display device 116. Of course, other output devices, such as printers, plotters, and the like, may alternatively be used in place of a display device 116. In FIG. 1 the graphics interface driver 112 and graphics interface 114 are displayed together indicating a tightly coupled relationship between the two.
Given a set of generic instructions, the device-specific instructions output by an updated graphics rendering engine could vary substantially from the device-specific instructions output by the previous graphics rendering engine. Such differences could be the result of more efficient processing of the generic instructions, improved support for advanced features provided by the graphics interface 114, and the like. Accordingly, one cannot simply compare the output of the updated graphics rendering engine to the output of the prior version. Unfortunately, this means that the only way to evaluate whether there is regression in the updated graphics rendering engine is for a user to perform a side-by-side visual inspection of a computer with an updated graphics rendering engine and another, similarly configured computer, with the previous graphics rendering engine, as both execute the same instructions.
FIG. 2 is a pictorial diagram illustrating an exemplary comparison system 200, as found in the prior art, for determining whether there is regression in an updated graphics rendering engine. More particularly, the exemplary comparison system 200 includes two similarly configured computers, a test computer 202 and control computer 204. The computers are configured as similarly as possible except that the test computer 202 includes an updated version of the graphics rendering engine, and the control computer 204 includes the previous version of the graphics rendering engine. A third computer, the master computer 206, is connected to both the test computer 202 and the control computer 204, sends test instructions 208 to test and control computers, typically at the direction of the user/visual inspector. As the test instructions 208 are sent to the subject computers, the user visually compares the results of the test instructions 208 as rendered on the computer monitors. When differences arise, they are noted by the user, typically with information corresponding to which test instructions have been executed thus far.
As those skilled in the art will appreciate, visual inspection is full of problems. User inspection is inefficient and can only be performed for short periods due to user fatigue. Users are also prone to miss differences, especially if they are subtle or fleeting in nature.
In light of the issues described above, what is needed is a system and method that automatically determines whether there is regression in an updated graphics rendering engine in comparison to a prior version of the graphics rendering engine. The present invention addresses this and other issues found in the prior art.